Synthesizing Light Field Video from Monocular Video

AbstractThe hardware challenges associated with light-field (LF) imaging has made it difficult for consumers to access its benefits like applications in post-capture focus and aperture control. Learning-based techniques which solve the ill-posed problem of LF reconstruction from sparse (1, 2 or 4) views have significantly reduced need complex hardware. video poses a special challenge as acquiring ground-truth training these models is hard. Hence, we propose self-supervised learning-based algorithm monocular videos. We use geometric, photometric temporal consistency constraints inspired recent technique stereo video. Additionally, three key that are relevant our input. an explicit disocclusion handling encourages network information adjacent input frames, inpainting disoccluded regions frame. This crucial single frame does not contain any about regions. also adaptive low-rank representation provides significant boost performance by tailoring each scene. Finally, novel refinement block able exploit available image data using supervised learning further refine quality. Our qualitative quantitative analysis demonstrates significance proposed building blocks superior results compared previous state-of-the-art techniques. validate reconstructing videos acquired commercial GoPro camera. An open-source implementation (https://github.com/ShrisudhanG/Synthesizing-Light-Field-Video-from-Monocular-Video).KeywordsLight-fieldsPlenoptic functionSelf-supervised